Finding the Boundary of Use to Improve Repair Mechanisms for Supervised Learning Systems

 Numerous “repair” mechanisms have been developed to improve the training data for supervised learning (SL) systems including feature selection, noise correction, and active learning. These repair mechanisms myopically repair instances as long the estimated system performance continues to improve. Such general repair can lead to unnecessary repairs and overfitting from repair which can lower system performance on new instances. We propose a Boundary of Use (BoU) meta-reasoning framework to decide which instances should be repaired. This framework uses a semi-supervised clustering approach to partition the training instances into regions where the SL system does well without repair, regions where it makes some mistakes, and regions where repair is deemed hopeless. Repair is then applied selectively to only mixed regions. We demonstrate that BoU-enhanced versions of repair improve SL system performance on 21 UCI datasets where general repair has varying degrees of unnecessary repair and overfitting.